DOCUMENT RESUME 



ED 470 164 



IR 021 604 



AUTHOR 

TITLE 

PUB DATE 
NOTE 



PUB TYPE 
EDRS PRICE 
DESCRIPTORS 



Koszalka, Tiffany A.; Grabowski, Barbara L . ; Kim, Younghoon 
Kids as Airborne Mission Scientists: Designing PBL To Inspire 
Kids. 

2001 - 11-00 

8p.; In: Annual Proceedings of Selected Research and 
Development [and] Practice Papers Presented at the National 
Convention of the Association for Educational Communications 
and Technology (24th, Atlanta, GA, November 8-12, 2001) . 
Volumes 1-2; see IR 021 504. 

Reports - Descriptive (141) -- Speeches /Meeting Papers (150) 
EDRS Price MF01/PC01 Plus Postage. 

^Educational Resources; Elementary Secondary Education; 
Instructional Design; Instructional Development; 

Instructional Innovation; Instructional Materials; ^Learning 
Activities; * Problem Based Learning; Science Achievement; 
^Science Activities; ^Science Education; ^Science Interests; 
Science Materials 



ABSTRACT 

Problem-based learning (PBL) has great potential for 
inspiring K-12 learning. KaAMS, a NASA funded project and an example of PBL, 
was designed to help teachers inspire middle school students to learn 
science. The students participate as scientists investigating environmental 
problems using NASA airborne remote sensing data. Two PBL modules were 
developed that address two different environmental issues: active lava flows 
and coral reefs in Hawaii. The modules consisted of new lesson plans that 
could be used flexibly by many teachers, and that use existing NASA and other 
Web resources. The goal was to harness those resources that exist rather than 
create totally new ones. This paper provides an overview of the instructional 
design challenges in creating Web-based PBL teacher support materials, the 
PBL model selected for use, the activities embedded in the PBL, and initial 
results of classroom trials. (Contains 10 references.) (Author/AEF) 
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Problem-based learning (PBL) has great potential for inspiring K-12 learning. KaAMS, an example of 
PBL, was designed to help teachers inspire middle school students to learn science. The kids participate as 
scientists investigating environmental problems using NASA airborne remote sensing data. This paper 
provides an overview of the instructional design challenges in creating web-based PBL teacher support 
materials, the PBL model we selected, the activities embedded in the PBL, and initial results of classroom 
trials. 

Introduction 

Kids can be motivated and inspired by making direct contributions to solving real scientific issues. Can 
teachers be inspired too? Through a PBL approach, KaAMS, a NASA funded project guides teachers to take middle 
school children on live and past NASA airborne missions to collect data to study two environmental issues. The 
ultimate mission of the project was to inspire kids to learn and develop a career interest in science, math, technology 
and geography by their particip ating as scientists in activities punctuated by “bursts” of interactive events 
culminating in the analysis of data from NASA airborne missions. The mission is accomplished by providing 
resources to teachers to use with middle school children. To do this, we developed two problem based learning 
modules addressing two different environmental issues — active lava flows and coral reefs in Hawaii. The modules 
consisted of new lesson plans that could be used flexibly by many teachers, and that use existing NASA and other 
web resources. The goal was to harness those resources that exist rather than create totally new ones. 

Problem-Based Learning Literature — the basis for KaAMS 



The conceptual framework of problem-based learning model for KaAMS is based on the perspectives and 
implications of problem-based learning literature. Problem-based learning as an instructional model associated with 
the new learning paradigm (e.g., Reigeluth, 1999) has been implemented in diverse content domains such as medical 
education, business education, social education, and science education. Problem-based learning, in general, 
encourages students to engage in learning activities to solve a real world problem (Duffy & Cunningham, 1996). 
According to problem-based learning researchers (Barrows, 1986, 1992; Savery & Duffy, 1995; Schwartz, et al, 
1999), key characteristics of problem-based learning include the following; 

• Real-world problems with a motivational context to drive learning. A real world problem is used as a 
stimulus for authentic activity. 

• Given a problem space, students generate their own learning goals in terms of what they will attain. 

• Multiple learning resources including print, electronic and humans are provided for the student to develop a 
deep understanding about content knowledge related to the problem and apply that knowledge into the 
problem solving activities. 

• Students as active problem solvers work with their peers, teachers, and experts to share their different 
perspectives and develop deeper knowledge on a subject area. 

• By placing students in learning by doing situations, students develop a disciplinary knowledge base, 
problem solving skills, reflective thinking skills, and collaboration and communication skills. 

• Teachers play a role as coach or facilitator that supports students’ learning and problem solving activity, 
rather than directly teaching what students should know and how students should solve a problem. 
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In general problem based learning (e.g., Barrows, 1986, 1992; Savery & Duffy, 1995; Schwartz, et al, 1999), 
students engage in the following five stages of a learning process: 

• Present a problem — Students start with a presentation of a real world problem. 
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• Generate what students know and what they need to know — Students actively define problems and generate 
what they know and what they need to know based on their prior knowledge and experience. They are 
encouraged to identify learning issues or knowledge necessary to construct an understanding about how to 
solve problems. 

• List possible actions— Based on the previous activity, students discuss and come up with strategies and 
activities for solving the problem. 

• Collect and analyze information— All students engage in gathering information from available learning 
resources ranging from print-based materials, electronic and human resources and from the designated 
facilitator. After gathering the information, they analyze and evaluate information in terms of what is most 
useful or what is not useful to solve the problem. They discuss and negotiate their perspectives about 
alternative solutions with peers, their teacher, and experts. 

• Present and share solutions— They finally propose their solutions, share them with their peers and experts 
who might provide different perspectives to the solutions, and revise their solution based on feedback from 
their peers or experts. 

The KaAMS Model 

From these perspectives and implications of the problem-based learning literature, we developed the 
KaAMS problem based learning model with four learning stages in which middle school students engage: 1) 
problem scenario, 2) propose ideas/search information, 3) conduct mission/collect and analyze data, and 4) propose 
solution. Each learning stage of the KaAMS problem based learning model includes the following key attributes: 

• Authentic, ill -structured problem situation 

• Assumption of roles by the students 

• Reflections about what they know, what they need to know 

• Planning the investigation procedure 

• Access to rich NASA web resources 

• Active investigation 

• Learning activities situated within real NASA missions 

• Reflective thinking exercises 

• Peer and expert collaboration 

• Learner activities/tools in interpreting data gathered 

• NASA scientist support 

• Shared solutions with peers and experts 

Conceptual Framework for KaAMS 
Foundation of KaAMS 

Conceptually, the KaAMS framework, as shown in Figure 1 , is built upon the premise and foundation that 
among all NASA web resources from all aspects of the agency, a multitude of resources can be used in the 
classroom. These resources are filtered through a second -level premise, which is the Web-Enhanced Learning 
Strategies (WELES) interface. This interface helps to sift through the available resources for elements and 
composite sites that are appropriate for use by middle school teachers and students. These resources are then used in 
four parts of a lesson plan — frame/inform/explore/try. The third premise is that teachers can use real web resources 
from real NASA missions in a problem based lesson format. Finally, these three levels of resources are harvested as 
part of the KaAMS PBL lesson plans. 

Students are presented with an environmental problem that is of concern of to a NASA mission. They begin 
a series of problem-solving lessons from which they develop content and applied knowledge by participating in 
problem solving activities. Through a series of framing and informing activities, students search for additional 
information on the problem, develop an understanding of the science of the problem, and propose a solutions for 
conducting a mission that will provide remote sensing data to solve the problem. Students become involved in 
“bursts” of activities to conduct a mission, collect and analyze data. Finally, students summarize their findings in 
several different ways and go public to share what they have learned with classmate and/or other outside their 
classroom. One important note is that the students participate in reflective activities throughout the entire process. 

Developmentally Appropriate Lessons 

Also evident in the design were the following key characteristics we found about middle school students 
(From: This We Believe). 
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• Moving from concrete to abstract thinking 

• Curious on a wide range of topics, few of which are sustained 

• Prefer active over passive learning 

• Respond positively to participating in real life learning situations 

• Are inquisitive and challenge adults 

• Desire recognition 




•From concrete to i 
abstract thinking 
•Curious on a wide 
range of topics, few 
of which are 
sustained 
•Prefer active over 
passive learning 
•Respond positively! 
to participating in 
real life learning 
situations 

•Are inquisitive and 
challenge adults 
•Desire recognition 



Figure 1. KaAMS Foundation 
Links to National Standards 

To maintain the link to the National Standards, we have completed an analysis of the NSTA/NRC standards 
and the AAAS Project 2061 Benchmarks to target in the KaAMS Project. Each lesson plan links to the specific 
national education standards that might be satisfied by completing the lesson activities 
Flexibility 

Since flexibility is very important to maximize the usability of this site, we have designed the site for the 
teacher. The framework is constructed so that the teacher is in control of how much and what types of the available 
activities that his or her students actually see. He or she can start from Phase 1 and proceed to Phase 4, or he or she 
can just go the activities of Phase 3, for example. 

Mapping Design Attributes of KaAMs onto PBL and the Scientific Process 

Problem-based learning and the scientific process follow similar steps. This cross over made explaining 
the learning process to content experts very easy. See Table 1. Some examples of how the design attributes were 
built into the KaAMs is shown in the last column. 



Table l. Mapping KaAMS onto the PBL Process 



PBL Process 


Scientific Process 


KaAMS 


Design Examples 


Problem Clarification 


Identify Problem 


Problem Scenario 


• Two problems — 
finding lava flows for 
the Pacific Disaster 
Center, and 
determining if the 
coral reef need 
protection for a real 
Congressional 
Executive Order 


Plan Development 


Research ideas and 


Propose Ideas/Search for 


• Activity Sheets as 
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Develop plan for 
investigation 


Information 


Who, What, When, 
Where, Why and How 
questions to determine 
what the students 
know 

• Students complete 
reflection journals 

• Find information 
using existing NASA 
web resources 

• Participate in activity 
bursts to explore new 
concepts 


Collect Information 


Collect Data 


Conduct Mission and 
Collect Data 


• Students think 
critically about which 
aircraft can run their 
mission and select 
from several 
possibilities 

• Students plan the 
actual mission and 
compare it to the 
actual NASA mission 


Analyze Information 


Analyze Data 


Analyze Data 


• Students use actual 
data from the live 
missions to draw 
conclusions 

• Students use 
guidelines from 
NASA scientists for 
interpretation of their 
data 


Present Solutions 


Report Findings 


Propose Solution: 
(Go Public) 


• Write a report to the 
Pacific Disaster 
Center, 

• Make 

recommendations to 
the President in 
response to his 
executive order 



Alpha Testing Phase 

The assessment strategy for the entire KaAMS project was divided into three major phases designed to 
capture data that would support initial product development (alpha testing), on-going resources development and 
implementation planning (beta testing), and analysis of impact on the stakeholders in the learning environment 
(research-impact analysis). See Figure 2. The diagram below illustrates the flow of alpha, beta, and research 
processes used for each of the two major products being developed; (1) volcano mission and (2) coral reef mission. 
This report summarizes overall data collection methods and procedures as well as the findings from the alpha 
development cycle for Products 1 and 2 - Mission: Studying active lava flows. 



ALPHA TEST 



BETA TEST (new classrooms) 



IMPLEMENT 
(new classrooms) 




RFST nnpv A\/A 1 1 API F 



Figure 2 . Overall Assessment and Research Strategy 
Methods 



During the Alpha testing formative and summative evaluation data were collected. With project 
enhancement in mind, data collected from key stakeholders during the alpha (initial formative development) testing 
phase included five levels of assessment: (1) reaction, (2) learning gains, (3) performance, (4) education system 
changes, and (5) impact on the greater society. Research protocols were also tested to assess their effectiveness in 
measuring the effects of KaAMS materials on teachers, students, and stakeholders in the surrounding community, 
namely parents. See Table 2. 

T able 2: Research questions and assessment 

Research Questions Assessment 

How are teachers using KaAMS and NASA resources? Performance 

How are teachers changing their teaching practices (e.g. teaching strategies, Performance/ 
incorporation of NASA resources, etc.) over time as a result of using System Change 

KaAMS and NASA resources? 

How are student levels of interest in pursuing science-related career Learning Gains 

changing over time as a result of using KaAMS and NASA resources? 

How does the use of KaAMS diffuse to the surrounding school system? Impact on Greater 

Society 

All formative evaluation instruments were administered throughout the alpha testing phases to gather 
feedback from teachers while preparing for and using the KaAMS materials. Interviews, observations, and focus 
groups were also conducted at least once per week with teachers and students during the 6-month alpha classroom 
trials. The summative instruments were administered to teachers at the end of the KaAMS classroom trials and final 
interviews and focus groups were conducted with teachers and students. The research instruments were administered 
to teachers and students for pre- and post-test data collection and at and additional 1 -month follow-up period for 
students. Parents were surveyed at the beginning and end of the school year. 

Subjects 

Three middle schools in a rural Pennsylvania school district participated in the KaAMS alpha test 
classroom trials, East, West, and Distant. Six different classrooms from these schools were actively involved. Four 
were 6 th grade, one was 7 th grade, and one was an 8 th grade honors class. The six teachers who participated provided 
data about themselves, their classrooms, and success of using KaAMS materials during the alpha testing cycle. 
Teaching experience ranged from 3 to 23 years; initial preferences for primary teaching strategies included hands-on 
activities, collaborative activities, role play, and problem-based learning; half of the teachers had moderate success 
using web resources in their classrooms the other half had not used such resources in their classrooms. 

Data were collected from a total of 144 students, 82 were boys, 59 girls and 3 did not respond to the gender 
question. On average, the students had a moderate level of interest in pursuing science. 

One hundred and fifty three parents of KaAMS students returned surveys indicating their initial perceptions of 
science in their school and child’s success and interest in science as well as reporting their highest attained level of 
education. A majority of the parents did not have college degrees, worked in non-science related jobs, and had a 
neutral opinion of their child’s school’s science program. 

Measures and Instruments 

Formative and Summative evaluation: A series of instruments, observation protocols, and interview 
protocols were developed to collect formative and summative data from the teachers and students during the alpha 
testing development cycle. 

Teachers were asked to review the KaAMS lesson plans, prepare to use the lesson plans in their 
classrooms, and complete evaluation surveys after each lesson and at the end of the trial indicating ease of use; value 
of resources, instructions, and assessment guidelines provided; success of activities; amount of preparation time; 
descriptions of the classroom activity during KaAMS lessons; and general feelings about using KaAMS for teaching 
and learning. Teachers were also asked to share feedback during interviews and focus groups including responses to 
questions such as: What did you like/not like about the supporting website? What parts of the lesson plans did you 
use - why? What additional support materials did you need to use these materials? What additional materials did the 
students need? What would you change? 
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Periodically students were asked, during interviews and focus groups, to respond to questions such as: 
What did you like/not like about the KaAMS activities? How useful were the internet sites? What was happening in 
the classroom during KaAMS? What did you learn? and what would have made these activities more useful to you? 
Observational data were collected several times during the classroom trial that lasted between 3 and 6 months, 
depending on the classroom teacher. Observation data were collected on how the teachers used the materials, how 
the students participated in the activities, and artifacts developed by the teacher or students during the KaAMS 
lessons. 

Research: The research questions were focused on the teachers, students, and parents. Teachers completed 
an on-line instrument eliciting background information, preferences for classroom activities, and attitudes toward the 
use of web resources in the classroom. The instrument was a combination of an attitude survey previously developed 
and validated for similar research (Koszalka, 2000), a series of questions related to perceptions of their school’s 
ability to support the use of internet technology in the classroom (McCarthy, Grabowski, & Koszalka, 1998), and 
preferences for teaching styles (Grabowski, Koszalka, & McCarthy, 2000; Koszalka, Grabowski, & McCarthy, 

2000). This instrument was administered at the beginning of the classroom trial period and the end (pre-post test). 

Data were collected on student level of career interest in science, pre-, post, and 1 -month after using 
KaAMS materials. Student career interest surveys were purchased from the APA. The survey also included a series 
of questions developed to assess reflective thinking (Koszalka, et al., 2001) and gather demographic data. 

Parents were asked to complete surveys at the beginning and end of the school year to assess their 
perceptions of their child’s school’s science program. The questions were taken from previous research on 
measuring parents’ perceptions of school programs. 

Results 



The initial formative feedback provided guidance in designing support structures for the KaAMS website 
that helped the alpha teachers connect NASA science to their curriculum and prompt active student involvement, as 
scientists, during science class. The results from the formative and summative evaluation resulted in: development 
of enhanced lesson plan structures for the KaAMS website, new content support for teachers that strengthened the 
relationship between the overall problem scenario and learning activities, further instructions to ‘coach’ teachers in 
using PBL, web technology, and activities that prompt student reflection, stronger ties between lesson plans and 
national education standards and curriculum requirements, and enhanced activities that will better meet kids’ needs. 
The initial research findings from the pilot classrooms were very encouraging. Although caution is warranted in 
interpreting these results, analysis of the research data collected during the alpha testing cycle showed significant, 
yet minor changes in teachers, students, and parents after the use of KaAMS in the classroom. Table 3 summarizes 
research findings in accordance with the KaAMS project research questions: 

Table 3. Research questions and Alpha Preliminary Findings 

Research Questions Alpha Preliminary Findings 

How are teachers using KaAMS and NASA Teachers noted the flexibility of KaAMS resources and used them 

resources? in a variety of ways to enhance or change the way they teach. 

How are teachers changing their teaching Several of the teachers tried new ways of integrating the web and 

practices over time as a result of using collaborative activities into their teaching; changed their preferred 

KaAMS and NASA resources? method of teaching and the types of resources they used regularly 

in their classrooms, and their attitudes toward using web 

resources in the classroom. 

How are student levels of interest in pursuing Significant increases in student level of science career interest 
science-related career changing over time 

as a result of using KaAMS? 

How does the use of the KaAMS products Parent perceptions of their child’s school’s emphasis on science, 

diffuse to the surrounding school system? school’s ability to provide good science experiences, and use of 

appropriate science resources were higher at the end of the school 
year than in the beginning. 

We believe that we are providing teachers with a venue and structure for using NASA web-based materials 
in their classroom in meaningful and contextualized ways that will support student knowledge development in the 
content and processes of science. Through their high quality materials, NASA can make an impact on science in the 
classroom, which in combination with KaAMS strategies can change teaching practice, impress middle school kids 
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with the importance of and strategies for conducting good science — the ultimate goal being to influence career 
aspirations of these kids toward science. 
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